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Abstract 

We present a new combinatorial characterization for local optimality of a codeword in an 
irregular Tanner code. The main novelty in this characterization is that it is based on a linear 
combination of subtrees in the computation trees. These subtrees may have any degree in the 
local code nodes and may have any height (even greater than the girth). We expect this new 
characterization to lead to improvements in bounds for successful decoding. 

We prove that local optimality in this new characterization implies ML-optimality and LP- 
optimality, as one would expect. Finally, we show that is possible to compute efficiently a 
certificate for the local optimality of a codeword given an LLR vector. 

1 Introduction 

Modem coding theory deals with finding good codes that have efficient decoders (see e.g. URUOSII ). 
Many of the decoders for modem codes are sub-optimal in the sense that they may fail to correct 
errors that are corrected by maximum likelihood (ML) decoder, but their simplicity and speed make 
them attractive in practice. Message-passing decoding algorithms based on belief-propagation and 
linear-programming (LP) decoding are examples for such sub -optimal decoders. 

Tanner UTanSll introduced graph representations of linear codes. In the standard setting, check 
nodes compute the parity function. In the generalized setting, check nodes use a local error- 
correcting code. One may view a check node with a local code as a coalescing of multiple parity 
check nodes. Therefore, a code may have a sparser and smaller representation when represented as 
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a Tanner code in the generalized setting. An example of a Tanner code with a simple bit-flipping 
decoding algorithm was presented by Sipser and Spielman [ISS96II . 

Linear programming (LP) decoding was introduced by Feldman, Wainwright and Karger [|Fel03[ 
IFWKOSH for binary linear codes. LP decoding has been applied to several families of codes, among 
them RA codes, turbo-like codes, LDPC codes, and expander codes. This work is motivated by the 
problem of analyzing the probability of successful decoding using LP decoding for Tanner codes. 
There are very few works on this problem, and they deal only with specific cases. For example, 
Feldman and Stein IIFS05II analyzed special expander codes, and Goldenberg and Burshtein HGBlOl 
deal with repeat- accumulate codes. 

The combinatorial characterization of a decoding success is based on a test criterion that certi- 
fies the optimality of a codeword. That is, given a received word y and a codeword x, we consider 
a test that answers the questions: is x optimal with respect to yl and is it unique? We call these 
tests certificates for the optimality of a codeword. Bounds on the word error probability may be 
computed by analyzing the events for which certificates are provided. 

Wiberg [|Wib96l studied representations of codes using factor graphs. He used these represen- 
tations to analyze message passing decoding algorithms. The analysis uses minimal combinatorial 
structures (i.e., skinny trees) to characterize decoding errors when using message passing decoding 
algorithms. 

Koetter and Vontobel IIKV06n analyzed LP decoding of regular LDPC codes. Their analy- 
sis is based on decomposing each codeword (and pseudocodeword) into a sum of skinny trees 
with uniform vertex weights. Arora et al. I1ADS09II extended the work in I1KV06II by introduc- 
ing nonuniform weights to the vertices in the skinny trees. For a BSC, Arora et al. proved that 
local optimality implies both ML-optimality and LP-optimality. They used analysis techniques, 
similar to those used in density evolution analysis, to improve bounds on the probability of a de- 
coding error. This work was further extended in [HEIU to memoryless channels. The analysis 
in I1KV061 [ADS091 [HEl 1|| is limited to skinny trees, the height of which is bounded by a quarter 
of the girth of the Tanner graph. 

Vontobel HVonlOU extended the decomposition of a codeword (and pseudocodeword) to sub- 
trees of the computation tree. This enabled him to avoid the limitation of the height being bounded 
by the girth. The decomposition is obtained by a random walk, and applies to irregular Tanner 
graphs. 

Jian and Pfister [IJPIOII analyzed a weighted min-sum decoding algorithm for regular LDPC 
codes. They used skinny trees in the computation tree, the height of which is greater than the 
girth of the Tanner graph. They also used local optimality to connect successful decoding to LP- 
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decoding. 



Contributions. We present a new combinatorial characterization of local optimality for irregu- 
lar Tanner codes. This characterization uses subtrees in the computation tree in which the degree 
of local code nodes is not limited to 2 (as opposed to skinny trees in previous analyses). Since 
such trees are bigger, it is likely that this characterization will lead to improved bounds for suc- 
cessful decoding. We prove that local optimality in this characterization implies ML-optimality 
and LP-optimality, as one would expect. Finally, we show that is possible to compute efficiently a 
certificate for the local optimality of a codeword given an LLR vector. 

2 Preliminaries 

Tanner-codes and Tanner graph representation. Let G = (VVJ J,E) denote an edge-labeled 

bipartite-graph between a set of N vertices V = {fi, . . . , vn} called variable nodes, and a set of 

J vertices J = {Ci, . . . , Cj} called local-code nodes where degdCj) = nj. 

is an [uj, kj, dj] code, j E [J] } denote a set of J local-codes. We associate 
—j — J 

every local-code C G C with the respective local-code node Cj G J. The set E consists of 
edges {vi, Cj) such that variable Vi participates in local-code & . The labels {1, . . . ,nj} of the 
edges incident to local-code node Cj indicate the order of variable bit nodes in the corresponding 
local-code C'' . Let d* = miui^j^j dj denote the smallest minimum distance among the local codes. 

Let a word x = {xi, . . . ,xn) G denote an assignment to variable nodes in V. Let Vj denote 
the ordered set of variable nodes in MciCj) according to labels of edges incident to Cj. Denote 
by xv^ G the projection of the word x = (zi, . . . , xat) onto entries associated with Vj. 

The Tanner code C{G, C ) based on labeled Tanner graph G is the code of block length 

with codewords x G such that is a codeword in C' for every j G [J]. 

J J j 

Consider a Tanner code C{G, C ), where C = {C }jg[j]. We say that a word 

satisfies local-code C' if G C\ Denote by O the set of words x that satisfy the local-code 



C , i.e., = {x G F^ : xv^ G C }. The resulting code is the extension of the local-code 
C'' from length Uj to length A^. We denote the set of extended local-codes in by C"-^ . Clearly, 
C(G,C^) C CK It holds that 



LP decoding of Tanner codes. When transmitting over a discrete memoryless channel, the re- 
ceiver observes a measurement yi for every transmitted symbol Xj. In memoryless binary-input 




(1) 



iG[J] 
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output-symmetric (MBIOS) channels, the log-likelihood ratio (LLR) vector A G is defined 
by KiVi) — In ( p|^Yx'=i) ) every input bit i. For a linear code C, Maximum-Likelihood (ML) 
decoding is equivalent to 

x'^^{y) = aTg min {X{y),x), (2) 

a;Gconv(C) 

where conv(C) denotes the convex hull of the set C. 

Solving in general the optimization problem in (|2]) for linear codes is intractable. Feldman et 
al. [|Fel03[ IFWK05II introduced a linear programming relaxation for the problem of ML decoding 
of Tanner codes whose local codes are parity codes. LP decoding is based on minimizing an 
objective function over a fundamental polytope defined by a Tanner graph G. A natural extension 
of LP decoding to Tanner codes in the generalized setting is obtained by optimizing the objective 
function over a generalized fundamental polytope. Consider a Tanner code C = C{G, C ). The 
generalized fundamental polytope V = V{G, C ) is defined as the convex hull 

V= f] conv(C^). (3) 

Clearly, the generalized fundamental polytope V{G, C ) is a function of the (edge labeled) Tanner 

—J 

graph G and the set of local-codes C . Note that the representation of Tanner codes via Tanner 
graph and local codes is not unique. Different representations (G, C ) of the same Tanner code C 
yield different generalized fundamental poly topes V for the same code C. We note that for Tanner 
codes whose Tanner graphs have constant bounded right degree and a linear number of edges, the 
generalized fundamental polytope has an efficient representation. This family of codes is typically 
called generalized low-density parity-check codes. 

Given an LLR vector A for a received word y, LP-decoding consists of solving the following 
optimization problem 

x^^{y) = aTg min {X{y),x). (4) 

The difference between ML-decoding and LP-decoding is that the fundamental polytope V{G,C ) 
may strictly contain the convex hull of C. Vertices of V{G, C ) that are not codewords of C must 
have fractional components and are called pseudocodewords. 

We now introduce some graph terminology. Let Mciv) denote the set of neighbors of node v 



in graph G, and for a sot S C V let AfdS) = [j^^s^G{v) 



between nodes v and u in G. Let dcir, v) denote the distancqj between nodes r and v in G. 



Let Pyu{G) denote a shortest path 



'Length of a shortest path 
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An induced subgraph is a subgraph obtained by deleting a set of vertices. The subgraph of 
G = {V, E) induced by S (^V, denoted by Gs, consists of S and all edges in E, both endpoints 
of which are contained in S. For a codeword x E C{G) C {0, 1}^, let Gx denote the subgraph of 
the Tanner graph G induced by \4 U MiVx) where Vx = {vi\ Xi = 1}. 

3 A Combinatorial Certificate for an ML Codeword 

In this section we present combinatorial certificate, that applies both to ML-decoding and LP- 
decoding, for codewords of Tanner codes. A certificate is a proof that a given codeword is the 
unique solution of maximum-likelihood decoding and linear-programming decoding. The cer- 
tificate is based on combinatorially structured weighted local configurations in the Tanner graph. 
These local configurations generalize the minimal configurations (skinny trees) presented by Von- 
tobel HVonlOII as extension to Arora et al. [IADS09I . We note that for Tanner codes, the support 
of each weighted local configuration is not necessarily a local valid configuration. For a given 
codeword, the certificate is computed by a message-passing algorithm on the Tanner graph of the 
code. 

Notation: Let y E R" denote the word received from the channel. Let A = X{y) denote the 
LLR vector for y. Let G = (V U J', E) denote a Tanner graph, and let C{G) denote a Tanner code 
based on G with local minimal distance d*. Let x E C{G) be a candidate for x^'^{y) and x^^{y). 

Definition 1 (Path-Prefix Tree). Consider a graph G = {V, E) and a node r E V. Let V denote 
the set of all backtrackless paths in G with length at most h that start at node r, and let 

E = {{pi,P2) eV xV \ piisa prefix of p2, + 1 = |p2|}- 

We identify the empty path in V with r. Denote by Tj^ {G) = {V,E) the path-prefix tree of G 
rooted at node r with height h. We denote the fact that a path p E V ends at v E V, by p ^ v. 

The path-prefix tree is constructed by recursively unwrapping graph G from a root node v for 
h iterations. When dealing with the analysis of belief propagation algorithms on graphical models, 
the path-prefix tree of a Tanner graph G rooted at a variable node is usually referred to as the 
computation tree. We make the distinction between the computation tree and the path-prefix tree 
since we consider also path-prefix trees of subgraphs of a Tanner graph G and are not necessarily 
rooted at a variable node. We denote vertices in the path-prefix tree by v,u, etc. Vertices in G are 
denoted by t;, n, etc. 
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The following definitions expands the combinatorial notion of minimal valid deviations [|Wib96ll 
and weighted minimal local-deviations (skinny trees) ||ADS09[ IVonlOII to the case of Tanner codes. 

Definition 2 (rf-tree). Consider a Tanner graph G = {VVJ J , E). A d-tree, T[r, h, d] {G), of height 
h rooted at node r is a subtree ofT^iG) such that every variable node has full degree and every 
local-code node has degree d. 

Definition 3 (w-weighted subtree). Consider a Tanner graph G = {VVJJ.E). Let% = (VLiJ^E) 
denote a subtree of%^{G), and let u = (wi, . . . , Uh) G R'j: denote a non-negative weight vector 
Let Tf'^'^ : V\{?^} —J" R denote a weight function for variable nodes in Tf as follows. 

where t = [^^^] and v v. Let Tf'^^ also denote the u-weighted subtree Tf rooted at node r. 

For any cci-weighted subtree T}'^^ of Tl^{G), let TiG\Tf'^^] G R'^' denote the projection of T}"^^ 
to the Tanner graph G. That is, for every variable node v in G, 

t^gW ](^) = \ (6) 

otherwise. 

For two vectors x E {0, 1}^ and / G [0, 1]^, let a: © / G [0, 1]^ denote the relative point 
defined by (x © /)» = \xi — fi\ flFelOBII . The following definition is an extension of local- 
optimality nADS09[|VonlOII to Tanner codes on memoryless channels. 

Definition 4 (local-optimality). Let C{G) C {0, 1}^ denote a Tanner code with minimal local- 
distance d*, and let uj G [0, 1]^\{0^} denote a non-negative weight vector of length h. For any 
integer 2 ^ i ^ d*, let B^^^ denote the set of all vectors corresponding to projections by u- 
weighted i-trees to G, i.e., Bf''^ = {7rG[T^'^^r,2h,i]{G)] \ r is a variable node in G}. A codeword 
X G {0, 1}^ is {h, u, z)-locally optimal for A G if for all vectors /3 G Bf^\ 

(A,x©/3) > (A,x). (7) 

Note that ^ C [0, 1]^ for every weight vector u G [0, 1]^. Based on random walks on the 
Tanner graph, Vontobel showed that {h,uj, 2) -local optimality is sufficient both for global opti- 
mality and LP optimality. The random walks are defined in terms derived from the generalized 
fundamental polytope. We extend the results of Vontobel HVonlOU to "thicker" skinny-trees by 
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using probabilistic combinatorial arguments on graphs and the properties of graph cover decod- 
ing IIVK05II . Specifically, we prove that {h, u, z)-local optimality, for any 2 ^ i ^ d*, implies LP 
optimality (Theorem|7]). Given the decomposition of Lemma [8] proved in SectionlH the following 
theorem is obtained by modification of the proof of [IADS091 Theorem 2] or [|HE11[ Theorem 6]. 

Theorem 5 (local-optimality is sufficient for ML). Let C{G) denote a Tanner code with minimal 
local-distance d*. Let h be some positive integer and u = {ui, . . . , Uh) G [0, 1]'* denote a non- 
negative weight vector Let X G denote the LLR vector received from the channel, and suppose 
that X is an {h,oj,i) -locally optimal codeword for A and some 2 ^ i ^ d*. Then x is also the 
unique maximum-likelihood codeword for A. 

Proof. We use the decomposition proved in Section |4] to show that for every codeword x' ^ x, 
(A, x') > (A, x). Since z = x (B x' is a codeword, by Lemma[8]there exists a distribution over the 
set bI'^\ such that E^^^c^)/? = az. Let / : [0, 1]^ — )■ E be the affine linear function defined by 

f{u) ^{X,x(Bu) = (A,'x) + Eti(-l)"'AiM,. Then, 

{X,x) < E^gg(^) (A, X © /3) (by local-optimality of x) 

= (A,x©E/3) (by linearity of / and linearity of expectation) 

= (A, X © az) (by Lemma[8]) 

= (A, (1 — a)x + a{x © z)) 

= (A, (1 — a)x + ax') 

= {1 — a){X,x) + a{X,x'). 

which implies that (A, x') > (A, x) as desired. □ 

In order to prove a sufficient condition for LP optimality, we consider graph cover decoding 
introduced by Vontobel and Koetter HVKOSn . We note that the characterization of graph cover 
decoding and its connection to LP decoding HVKOSn . can be extended to the case of Tanner codes 
in the generalized setting. We use the terms and notation of Vontobel and Koetter HVKOSn in 
the statement of Lemma [6l The following lemma shows that local-optimality based on z-trees is 
preserved after lifting to an M -cover. Note that the weight vector must be scaled by the cover 
degree M. 

Lemma 6. Let C{G) denote a Tanner code with minimal local-distance d*, and let G denote any 
M -cover of G. Let uj G [0, for some positive integer h. Suppose that x G C{G) is an 

{h, u,i) -locally optimal codeword for X G Ji^ for some 2 ^ i ^ d*. Let x = x^^^ G C{G) and 
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\ = X^'^'^ g R^'^^ denote the M-lifts of x and A, respectively. Then x is an {h, M ■ u, i)-locally 
optimal codeword for A. 

Proof Assume that x = x^*^ is not a {h, M ■ u, z)-locally optimal codeword for A = A^^^. Then, 
there exists an z-tree T = T[f, h, i]{G) rooted at some variable node f G V and a weight vector u, 
such that the projection /3 = n^[T^^^-'^^ G [0, l]^'^^ of the (M ■ a;)-weighted z-tree T^^-'^) onto G 
satisfies 

(A,x©/3) ^ (A,x). (8) 
Note that for x G {0, 1}^-^^ and its projection x = p{x) G R^, it holds that 

^(A,£) = (A,x), and (9) 
-L(A,5©^) = (A,x©/3), (10) 

where /3 = ■kc[T'^'^'^] G [0, 1]^ is the projection of the a;-weighted z-tree T onto the base graph 
G. From ([8]), ©, and (flOl) we get that (A, x) ^ {X,x (B /3), contradicting our assumption on the 
{h,uj, z)-local optimality of x. Therefore, x is a {h, M ■ cu, z)-locally optimal codeword for A in 
C{G). □ 

The following theorem is obtained as a corollary of Theorem |5] and Lemma [6l The proof is 
based on arguments utilizing properties of graph cover decoding. Those arguments are used for 
a reduction from ML-optimality to LP-optimality similar to the reduction presented in the proof 
of BHEni Theorem 8]. 

Theorem 7 (local optimality is sufficient for LP optimality). For every Tanner code C{G) with 
minimal local-distance d*, there exists a constant M such that, if 

1. iue [0, jj]''\{0^},and 

2. X is an {h, u, i)-locally optimal codeword for A G R^ and some 2 ^ i ^ d*, 
then X is also the unique optimal LP solution given A. 



3.1 Verifying local optimality 

Let G = {VUJ',E) denote a Tanner graph, and let C{G) denote a Tanner code with minimal local- 
distance d*. Let h denote a positive integer and u G [0, 1]^. Consider a codeword x G C{G) and 
any integer 2 ^ i ^ d*. Note that for a given LLR vector A, the weighted i-tree T^'^^ [r, h, i] (G) that 
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minimizes (A, x © /3) for all vectors /3 corresponding to projections of w-weighted z-trees rooted 
at r, can be computed by a simple message passing algorithm. The messages are propagated from 
the leaves of Tr '^\G) to the root r. In fact, message-passing algorithms on computation trees run 
simultaneously for every root in G. After h iterations we can verify if the codeword x is indeed 
{h, cu, z)-locally optimal for A (according to Definition IH). We can therefore compute an {h, co, i)- 
local optimality certificate in 0{\E\ ■ h) time. 

4 Constructing Codewords from Weighted Trees Projections 

This section features Lemma[8l which is the key structural lemma in the proof of Theorem |5] This 
Lemma shows that every codeword of a Tanner code can be constructed by a summation over a 
finite set of projections of weighted trees in the computation trees of G. 

Lemma 8. Let C{G) denote a Tanner code with minimal local-distance d*, and let h denote some 
positive integer. For every codeword x ^ 0^, and for every 2 ^ i ^ d*, there exists a distribution 
over i-trees T of G of height h and a positive integer H such that, for every weight vector uj G 
[0, -^]^\{0''}, there exists an a E (0, 1], such that 

E^gg(„) [hg\T\\ = ax. 

We first prove that every codeword x E C{G) can be decomposed into exactly weighted 
path-prefix trees (see Lemma |9]). Then we show that every weighted path-prefix tree can be de- 
composed to a set of weighted 2-trees (see Lemma [TOll. Putting these two results together yields 
Lemma [H 

Lemma 9. Let C{G) denote a Tanner code and let h denote some positive integer For every 
codeword x ^ 0^, and for every weight vector uj e R^J., 

t=l r:Xr=l 

Proof. Let us consider two variable nodes u,v E G^. Notice that \{v E l~^{Gx) '■ i) ^ v}\ = 
\{u E T^{Gx) ■ u u}\. Indeed, for every path from the root of T^{Gx) to a node v E {v : v 
v}, there exists a unique reversed path in T^{Gx) from the root to a node u such that u u. Let 
= {v, . . . ,f) denote a path in the path-prefix tree T^^ rooted at v, then = {r, . . . ,v) denotes 
the corresponding reversed path in the path-prefix tree Tj^. 
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Consider an all-one weight vector rj = l^. In O-CH), let Tr™ = Tr\G^), deg(-) = 
deg(^^(-), d{-,-) = dj-2h(^Q^){-, ■), r ~ r, and u u. Let qo p denote the concatenation of path q 
with path p. Equation (fTT)) holds for every 1 ^ i ^ 2h. 



{ j^'=(i),...,r):ci(-!;,r)=i} {"tf =(?;,..., it) :d(i',M)=i—l} {fi^N{u):d{v,f)=i} 



{"^=(?;,...,{i):d(?;,u)=i-l} {f<^J\f{u):d[v,f)=i} 



, degiu) — 1 

{"^=(?;,...,{i):d(s;,ji)=j-l} {f(iM{u):d{v,f)=i} ' 

E (11) 

Note that the reversed paths V ^^id V i^i the summations of (fTTI) end at a node {) such that 
V V. Equation (fTTI) implies that the sum of all ?7-weighted assignments to nodes {) ~ v in 
{Tr^^ (Gx) '■ Xr = 1} that correspond to paths of length i does not depend on i. 

In particular, for i = 1, J2{'f={v f)} = 1. It follows that for every 1^2^ 2h, 

E T}^\'v) = l. (12) 

={v ,. . . ,f):d{v ,f)=i\ 

Note that for every two variable nodes v, r, it holds that Tr'^\v) = 0Jd{r,v)/2 ■ Tr'^\v). Hence, 
Y.{-f={v,...,r):d{v,f)=2i} Tr'^\'j>) = ^i- Wc concludc that for every variable node v in 

h 

E ^[r/-)(G'.)]M = (E^O' (13) 

r:Xr=l i=l 

and the claim follows. □ 

Lemma 10. For every connected subgraph Gs of a Tanner graph G, let d denote the minimal 
degree of a local-code node in Gs- Then for every variable node r G Gs, a positive integer h, 
2 ^ i ^ d, and every weight vector u G R+, it holds that 



r}^\Gs) = E[r^^\r,2h,i]iG, 
with respect to a uniform distribution over i-trees T ofGs rooted at r with height 2h. 
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Proof. Consider a subgraph Gs of a Tanner graph G, and a positive integer i ^ d. Let %■ (Gs) 
denote an w-weighted path-prefix tree rooted at node r with height 2h. We want to show that the 
uniform distribution over w-weighted z-trees has the property that the expectation of trees over the 
distribution equals Tr'^^ (Gs)- 

We grow an i-tree rooted at r randomly in the path-prefix tree T^^{Gs)- That is, start from the 
root r. For each variable node take all it's children, and for each local-code node choose i distinct 
children uniformly at random. Let T[r, 2h, i] denote such a random z-tree, and consider a variable 
node i) E 'Tj?'^{Gs)- Note that T^'^^r, 2h, i]{v) is constant and does not depend on the random 
process. Equation (fT4)) develops the equality 

E[r(")[r,2/i,z]({))] =X^''\v). 



E[r(")[r,2/i,z](i))] = P(r[r, 2/1, z]) •r^^)[r,2/i,2]({;) 

{r[r,2fc,i]e7?MGs)} 

Y ¥{r[r,2h,i])-r^'^\r,2h,{\iv) 

{r[r,2/i,i]e7?''(Gg):i)er[r,2h,i]} 

= r^'^^[r,2h,i]{v) ■ J2 F{r[r,2h,i]) 

{rlr,2h,i]€T^'^iGs).veT[r,2h,i]} 

= T^'^^[r,2h,i]{v) ■¥{{! eT[r,2h,i]) 

= r^-\r,2h,z]{v). n . 1 

aeg{u) — 1 

«Gfri)\{'','«}nj7 

_ ^d{rfi)l2 -TT 1 TT % — \ 



degfO) ■ ii - l)'^(^^«)/2 n deg(M) - 1 ' 11 ^egfti) - 1 

'^d(r,«)/2 



n 



= r,("n^) (14) 

as required. □ 

5 Conclusion 

A new combinatorial characterization for local optimality of a codeword in an irregular Tanner 
code is presented. The main novelty in this characterization is that it is based on a linear com- 
bination of subtrees in the computation trees. These subtrees may have any degree i in the local 
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code nodes, for 2 < i < d*. This increased degree enables each subtree to be larger than a skinny 
tree. The larger a subtree is in the decomposition, the smaller the probability that its cost is nega- 
tive. Thus, we expect this new characterization to lead to improvements in bounds for successful 
decoding. 

It is interesting to develop and analyze decoding algorithms for irregular Tanner codes that are 
based on this new characterization of local optimality. 
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